SPORTS SIMULATION SYSTEM 



Field of the Invention 

[001] The present invention relates generally to entertainment systems and in 

particular to a sports simulation system. The present invention also relates to a 
projectile tracking apparatus for use in a sports simulation system to track accurately 
5 the path of a launched proj ectile. 

Background of the Invention 

[002] Sports simulation systems designed to simulate sports experiences are 

well known in the art. In many conventional sports simulation systems, a player 

10 propels a sports projectile such as a ball, puck, arrow, dart, etc. at a target image 
presented on a display screen. The motion of the sports projectile is detected and 
imaged and an extrapolation of the trajectory of the sports projectile is made. The 
extrapolated trajectory is then used to determine a sports result. The displayed image 
is in turn updated to reflect the sports result thereby to provide the player with visual 

15 feedback and simulate a sports experience. 

[003] The goal of all sports simulation systems is to provide the player with a 

realistic sports experience. As a result, many variations of sports simulation systems 
have been considered in attempts to simulate accurately "real-life" sports experiences. 
For example, U.S. Patent No. 5,443,260 to Stewart et al. discloses a baseball training 

20 and amusement apparatus that detects the speed and projected flight of a batted 

baseball. The apparatus includes a ball delivery device, a pair of detection planes, a 
computer and a video and simulation monitor. The detection planes are parallel to 
one another and are spaced apart by a distance such that a batted ball passing through 
the detection planes would be a fair ball in a real baseball game. Each detection plane 

25 includes a rigid frame that supports a pair of optical scanners and a pair of light 

sources. The optical scanners and light sources are positioned at opposite top corners 
of the rigid frame and are aimed downwardly into the region encompassed by the 
frame. r 

[004] During use, the ball delivery apparatus delivers a baseball towards a 

30 player positioned in front of the detection planes. When the player strikes the 

baseball with a bat and the baseball travels through the detection planes, the optical 
scanners capture images of the baseball. The images are processed to determine the 



coordinates of the baseball as it passes through each of the detection planes as well as 
the velocity of the baseball. A simulated trajectory of the baseball is then calculated 
using the determined coordinate and velocity information. The simulated trajectory 
information is used to update the graphical images presented on the monitor so that 
5 the simulated flight of the batted baseball is displayed to the player thereby to 
simulate a batting experience. 

[005] U.S. Patent No. 5,649,706 to Treat, Jr. et al. discloses a hunting 

simulator for in-flight detection of a launched missile such as an arrow. The hunting 
simulator includes a screen and a projector for projecting a moving target on the 
10 screen. Electromagnetic radiation emitters are positioned in front of the screen 

adjacent its opposite top corners and illuminate a plane in front of the screen. Sensors 
are also positioned adjacent the opposite top corners of the screen and are responsive 
to the electromagnetic radiation emitters. Retroreflective tape extends along opposite 
sides of the plane. 

15 [006] During use, when an arrow is launched at the screen and passes 

through the plane, the sensors detect the presence of the arrow and generate output. 
The output of the sensors is used to determine the coordinates of the arrow as well as 
the velocity of the arrow. A simulated trajectory of the arrow is then calculated and 
the (graphical images presented on the screen are updated accordingly to reflect the 

20 flight of the launched arrow. In this manner, a hunting experience is simulated. 
[007] U.S. Patent No. 5,768,151 to Lowy et al. discloses a system for 

determining the trajectory of an object in a sports simulator. The system includes a 
baseball throwing device to deliver a baseball towards a player area. A projector 
adjacent the player area presents images on a display screen that is positioned near the 

25 ball throwing device and in front of a batter. Video cameras are positioned in front of 
and on opposite sides of the anticipated trajectory of a hit baseball. 
[008] During use when a baseball delivered by the ball throwing device is hit 

by the batter and passes through the fields of the view of the video cameras, images of 
the baseball are captured and a streak showing the path of the baseball through the 

30 fields of view is determined. The streak is used to simulate the flight of the baseball 
and to update the image presented on the display screen thereby to simulate a batting 
experience. 
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[009] Although the above references show sports simulation systems that 

capture images of launched projectiles and use the image data to simulate the flights 
of the launched projectiles, these sports simulation systems fail to provide "true to 
life" sports experiences as a result of the mechanisms used to track the path of the 
5 launched projectiles. As will be appreciated, improved sports simulation systems that 
provide better and more realistic sports experiences are desired. 
[010] It is therefore an object of the present invention to provide a novel 

sports simulation system and a novel projectile tracking apparatus. 

10 Summary of the Invention 

[011] Accordingly in one aspect of the present invention there is provided a 

sports simulation system comprising: 

a projectile tracking apparatus including a display surface on which a 

visually apparent three-dimensional sports scene is presented, said projectile tracking 
1 5 apparatus capturing images of a projectile tracking region disposed in front of said 

display surface to detect a launched projectile traveling through said projectile 

tracking region towards said display surface; and 

at least one processing stage receiving the image data and determining 

the three-dimensional positions, velocity and deceleration/acceleration of a detected 
20 projectile traveling through said projectile tracking region, the three-dimensional 

positions, velocity and deceleration/acceleration being used by said at least one 

processing stage to calculate a trajectory of said launched projectile into said visually 

apparent three-dimensional sports scene. 

[012] Preferably, the at least one processing stage uses the calculated 

25 trajectory to generate updated image data including a simulation of the launched 
projectile into the visually apparent three-dimensional sports scene following the 
calculated trajectory. A display device coupled to the at least one processing stage 
receives the image data from the at least one processing stage and presents the 
visually apparent three-dimensional sports scene including the simulation on the 
30 display surface. It is also preferred that the visually apparent three-dimensional scene 
includes at least one foreground action element overlying and moveably over a 
background image. The at least one processing stage updates the image data so that 
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the foreground action element responds to the simulation of the launched projectile. 
Preferably, the at least one foreground action element, background image and 
launched projectile simulation are treated as non-destructive overlay image planes that 
are combined seamlessly to complete the visually apparent three-dimensional sports 
5 scene. 

[013] In one embodiment, the projectile tracking apparatus includes a frame 

encompassing a rectangular region and at least one pair of imaging devices mounted 
on the frame. The imaging devices have overlapping fields of view looking across 
and in front of the display surface from opposite corners of the rectangular region and 

10 capture images of the projectile tracking region. Preferably, each of the imaging 

devices includes a first processor generating two-dimensional projectile position data 
as the projectile travels through the projectile tracking region. The two-dimensional 
projectile position data generated by each first processor is conveyed to a host 
processor and used by the host processor to generate three-dimensional position data 

1 5 and to calculate the velocity and deceleration/acceleration of the projectile. 
[014] In a preferred embodiment, the host processor executes sports 

simulation software including at least two of a training mode, a practice mode and a 
game play mode. The visually apparent three-dimensional sports scene is consistent 
with the selected mode. It is also preferred that the sports simulation software 

20 includes a plurality of selectable sports modules, each relating to a different sport that 
can be simulated. Each sport module includes at least two of the training, practice 
and game play modes. 

[015] In another aspect of the present invention there is provided a sports 

simulation system comprising: 

25 a projectile tracking apparatus including a frame encompassing a 

display surface on which a video sequence portraying a visually apparent three- 
dimensional sports scene is presented; at least one pair of digital cameras mounted on 
said frame and having fields of view looking across and in front of said display 
surface that overlap in a generally perpendicular fashion and encompassing a 

30 projectile tracking region, each of said digital cameras including a first processor for 
processing image data and generating two-dimensional projectile coordinates when a 
projectile travels through said projectile tracking region and is captured in images 
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acquired by said digital cameras; and an audio system to broadcast audio 
accompanying said video sequence; 

a host processor communicating with said digital cameras and said 
audio system, said host processor calculating a three-dimensional trajectory of said 
5 projectile using the two-dimensional projectile coordinates received from each first 
processor and outputting image data including said calculated three-dimensional 
trajectory; and 

a display unit receiving said image data and presenting said video 
sequence including a simulation of said calculated trajectory on said display surface. 
1 0 [016] In yet another aspect of the present invention there is provided a 

projectile tracking apparatus for a sports simulation system comprising: 

a frame encompassing a display surface on which a video sequence 
portraying a visually apparent three-dimensional sports scene is presented; 

at least one pair of digital cameras mounted on said frame and having 
1 5 fields of view looking across and in front of said display surface that overlap in a 
generally perpendicular fashion and encompassing a projectile tracking region, each 
of said digital cameras including a processor for processing image data and generating 
two-dimensional projectile coordinates when a projectile travels through said 
projectile tracking region and is captured in images acquired by said digital cameras; 
20 and 

an audio system to broadcast audio accompanying said video sequence. 
[017] The present invention provides advantages in that since the positions, 

velocity and acceleration of the launched projectile are determined generally 
continuously as the launched projectile moves through the fields of view of the 

25 imaging devices, the flight of the launched projectile can be simulated more 

accurately. This of course results in a more realistic sports experience. The present 
invention also provides advantages in that virtually any type of projectile may be 
tracked avoiding the need for a specialized projectile. In addition, the projectile may 
be launched at the projectile tracking apparatus from a variety of positions providing 

30 greater flexibility and enhancing the sports experience. 

Brief Description of the Drawings 



[018] Embodiments of the present invention will now be described more 

fully with reference to the accompanying drawings in which: 

Figure 1 is a perspective of a sports simulation system in accordance 
with the present invention; 
5 , Figure 2 is a side elevation view of the sports simulation system of 

Figure 1 ; 

Figure 3 is a top plan view of the sports simulation system of Figure 1 ; 
Figure 4 is a front elevation view of a projectile tracking apparatus 
forming part of the sports simulation system of Figure 1 ; 
1 0 Figure 5 is an enlarged front elevation view, partly in section, of a 

portion of the projectile tracking apparatus of Figure 4 showing a digital camera and 
mirror and lens arrangement; 

Figure 6 is a perspective view showing the field of view of a digital . 
camera relative to a screen that form part of the projectile tracking apparatus of Figure 
15 4; 

Figure 7 is a schematic block diagram showing the electrical 
components of the sports simulation system of Figure 1; 

Figures 8 to 10 are flowcharts showing steps performed during player 
interaction with the sports simulation system of Figure 1 ; and 
20 Figure 11 is an enlarged front elevation view, partly in section, of an 

alternative digital camera and mirror arrangement for the projectile tracking apparatus 
of Figure 4. 

Detailed Description of the Preferred Embodiment 

25 [019] The present invention relates generally to a sports simulation system 

including a projectile tracking apparatus having a display surface on which a visually 
apparent three-dimensional sports scene is presented. The projectile tracking 
apparatus captures images of a projectile tracking region disposed in front of the 
display surface to detect a launched projectile traveling through the projectile tracking 

30 region towards the display surface. At least one processing stage communicates with 
the projectile tracking apparatus and is responsive to the image data to determine the 
three-dimensional positions, velocity and deceleration/acceleration of a detected 
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projectile traveling through the projectile tracking region. The determined three- 
dimensional positions, velocity and deceleration/acceleration are used by the at least 
one processing stage to calculate a trajectory of the launched projectile into the 
visually apparent three-dimensional sports scene. Updated image data is generated by 
5 the at least one processing stage that includes a simulation of the launched projectile 
into the visually apparent three-dimensional sports scene following the calculated 
trajectory. A display device coupled to the at least one processing stage receives the 
image data from the at least one processing stage and presents the visually apparent 
three-dimensional sports scene, including the simulation, on the display surface. 
10 Preferred embodiments of the present invention will now be described with reference 
to Figures 1 to 1 1 . 

[020] Turning now to Figure 1, a sports simulation system in accordance 

with the present invention is shown and is generally identified by reference numeral^ 
100. As can be seen, sports simulation system 100 includes a projectile tracking 

15 apparatus 102 disposed in front of a player area A in which a player P stands. A host 
computer 104 is coupled to the projectile tracking apparatus 102 via a high-speed 
serial data link and to a ceiling mounted front video projector 106 that is aimed at the 
projectile tracking apparatus 102. The computer 104 outputs video image data to the 
projector 106, which in turn projects a video sequence on the projectile tracking 

20 apparatus 102. The video sequence portrays a visually apparent three-dimensional 
sports scene including a target T at which a projectile is to be launched. 
[021] The projectile tracking apparatus 1 02 outputs two-dimensional 

projectile position data to the computer 104 when a projectile FB travels through a 
projectile tracking region monitored by the projectile tracking apparatus. The 

25 computer 1 04 in turn processes the two-dimensional projectile position data to 

determine the three-dimensional positions, velocity and acceleration of the projectile 
so that the trajectory of the projectile can be calculated. The calculated trajectory is 
then used to determine a sports result and to update the image data conveyed to the 
projector 106 so that the video sequence presented on the projectile tracking apparatus 

30 102 shows a simulation of the projectile travel into the visually apparent three- 
dimensional scene as well as the determined sports result. As a result, the projectile 
tracking apparatus 102, the computer 104 and the projector 106 form a closed loop in 
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response to a launched projectile. Further specifics of the sports simulation system 
100 will now be described with reference to Figures 1 to 10. 

[022] Figures 2 to 6 further illustrate the projectile tracking apparatus 102. 

As can be seen, the projectile tracking apparatus 102 includes an upright, inverted U- 
5 shaped frame 1 10 having a pair of side posts 112 and a crossbar 114 extending 

between the upper ends of the posts 112. A screen 122 is supported by the frame 110. 

Screen 122 includes a display surface 124 on which images projected by the projector 

106 are presented. In this embodiment, the screen 122 has a 4:3 aspect ratio making it 

particularly suited for displaying conventional television images. Those of skill in the 
10 art will however, appreciate that other image formats can be used. The screen 122 is 

preferably made of high strength, high durability nylon type material such as that 
. manufactured by Dupont under the name Cordura®. The screen 122 is loosely 

fastened to the back of the frame 1 10 at spaced locations. The bottom of the screen 

122 is free and is weighted. In this manner, the screen 122 absorbs impacts from 
1 5 launched projectiles thereby to reduce rebound potential. Preferably the frame 110 

can be easily disassembled and reassembled making the projectile tracking apparatus 

102 readily portable. v 

[023] Mirrors 128 and associated lenses 129 are positioned at each corner of 

the frame 110. Each mirror has a reflective surface that forms approximately a 22.5° 

20 angle with respect to the horizontal. The lenses 129 are formed of durable 

plastic/acrylic material such as for example Lexan®. An imaging device in the form 
of a two-dimensional monochrome CCD digital video camera 130 is associated with 
each mirror 128 and lens 129 arrangement. The digital cameras 130 are mounted 
within the side posts 112. The optical axis OA of each digital camera 130 is aligned 

25 with the center of its associated mirror 128 so that the optical focal axes of the digital 
cameras 130 are reflected at approximately 45° angles towards the center of the 
display surface 124. The mirrors 128, which fold the optics, allow the digital cameras 
130 to be positioned close to the projectile tracking region 120 while providing 
adequate focal distances to enable the digital cameras to capture focussed images. 

30 [024] The lens of each digital camera 130 in combination with the associated 

lens 129 provides each digital camera 130 with a viewing 1 angle equal to at least 70 
degrees and preferably between 75 to 85 degrees. The field of view FOV of each 
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digital camera 130 as reflected by the associated mirror 128 looks across and in front 
of the display surface 124 of the screen 122 and is generally pyramidal in shape as 
shown in Figure 5. The fields of view of the digital cameras 130 overlap in a 
generally perpendicular fashion to define a projectile tracking region or volume 120 
5 shown by the hatched lines in Figures 2 and 3. The lenses 129 also serve as protective 
covers to protect the mirrors 128 and inhibit debris from entering into the frame 110. 
Each digital camera 130 includes an on-board processor (not shown) to perform 
image data preprocessing as well as an IR pass filter (not shown). 
[025] In this embodiment, each digital camera 130 has a 640 by 480 pixel 

10 array and can be operated at very high frame rates thereby allowing multiple images 
of a fast moving projectile to be captured as it travels through the projectile tracking 
region 120. For example, in the case of a projectile travelling through the projectile 
tracking region 120 at a speed of 200 miles per hour, the frame rates of the digital 
cameras 130 are selected such that at least four images of the projectile are captured 

15 by each digital camera 130. The viewing angles of the digital cameras 130 and the 
dimensions of the frame 1 10 are selected to provide the digital cameras 130 with a 
resolving accuracy of approximately 1mm per pixel. As a result, a small projectile 
such as a golfball will activate approximately 12 pixels per image. This resolving 
accuracy enables even small, very fast moving launched projectiles to be readily 

20 determined in captured images and as a result, reduces false projectile detection. 

[026] Infrared (IR) light emitting diode (LED) arrays 132 are also positioned 

within the posts 122 beside the digital cameras 130. Each IR LED array 132 is 
associated with a digital camera 130 and emits IR radiation that is reflected by the 
associated mirror 128 into the projectile tracking region 120. Audio speakers 140 are 

25 provided on the posts 1 12 and are aimed forwardly toward the player area A. The 
audio speakers 140 are driven by an audio amplifier 142 accommodated within the 
frame 1 10 as shown in Figure 7. The audio amplifier 142 receives audio input from 
the computer 104 during play that is conveyed to the audio speakers 140 for broadcast 
thereby to enhance the sports experience. 

30 [027] The projector 1 06 preferably has a resolution of at least 800x600, at 

least 1200 ANSI Lumens brightness, a short throw lens, vertical 'keystone' 
correction, and the capacity to output digital RGB computer video signals, and 
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. NTSC/PAL baseband television video signals. Projectors having this set of features 
include the Epson Powerlite 820P, the Toshiba TDP-DI-US, the InFocus LP650 and 
the Sanyo XP30 for example. 

[028] The computer 104 is preferably an IBM compatible personal computer 

5 including an Intel Pentium® processor, at least 8 MB SDRAM, a high-speed hard 

drive, and a DVD player. The computer 104 also includes a display adapter assembly 
including a reconfigurable 32-bit video memory buffer partitioned into three separate 
buffers. One of the buffers is used to store primary foreground image data 
representing one or more independent foreground action elements. A second of the 

10 buffers is used to store background image data and the third buffer is used to store 
projectile trajectory image data. The display adapter assembly treats the foreground 
action, background and projectile trajectory image data as overlay image planes that 
are combined seamlessly to generate the video image data that is output to the 
projector 106. The overlay image planes are non-destructive so that when a 

1 5 foreground action element and/or projectile moves over an underlying image plane it 
is not necessary to redraw the underlying image plane. To reduce peak processing 1 
requirements, the computer 104 updates the background image data less frequently 
than the foreground image data. The computer 104 provides the output video image 
data to the projector 106 on a video output channel. The computer 104 receives 

20 external video feeds on a television/satellite/cable input channel, a video game input 
channel and an Internet input channel. 

[029] The computer 1 04 is mounted within a protective enclosure (not 

shown) having external connectors to enable the computer to be coupled to the 
projector 106, the audio amplifier 142 and the digital cameras 130. The enclosure 

25 also includes external connectors to allow the computer 104 to receive the 

television/satellite/cable, external video game and Internet feeds. Input controls are 
also provided on the enclosure to allow a player to interact with the computer 104. 
[030] The computer 104 executes sports simulation software stored in the 

SDRAM. The sports simulation software includes a plurality of sports simulation 

30 modules relating to sports that require a player to launch a projectile at the projectile 
tracking apparatus 102 in response to the video sequence displayed on the screen 122. 
To reduce memory requirements, the video image data associated with each video 
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sequence is stored in compressed form in the SDRAM and is decompressed on the fly 
during display. 

[031] In this particular embodiment, the sports simulation modules include a 

baseball module, a football module, a soccer module, a golf module and a hockey 
5 module. Each sports module includes three routines, namely a play game routine, a 
practice routine and a training routine. Each sports module also includes a 
characteristic projectile signature representing the projectile associated with the 
particular sport. The characteristic projectile signature is used to facilitate projectile 
detection during play as will be described. 

10 [032] During execution of the play game routine, the associated sport is 

simulated providing the player with a realistic sports experience. Skill levels, player 
preferences and statistics to be monitored can be set by the player prior to game play. 
The player can select whether to play against the computer 104, against another player 
or against an on-line player over an Internet connection. During execution of the 

15 practice routine, practice exercises are simulated. Again, skill levels, player 

preferences and statistics to be monitored can be set by the player prior to game 
practice. During execution of the training routine, training sessions are simulated. 
[033] The initialization and operation of the sports simulation system 100 

will now be described with particular reference to Figures 8 to 10. When the sports 

20 simulation system 100 is powered (step 300), the computer 104 performs a series of 
software diagnostics to set the environment and ensure that the projector 106, the 
digital cameras 130 and the audio amplifier 142 are functioning properly (step 302). 
A main menu 400 is then presented on the display of the computer 104 that allows the 
player to select an input video source (step 304). Specifically, the main menu 

25 includes a television/satellite/cable option 402, an external video game option 404, a 
sports simulation module option 406, an Internet option 408 and a system utilities 
option 410. When one of options 402, 404 and 408 is selected, the appropriate 
incoming video feed received by the computer 104 is used to generate the video 
image data that is output to the projector 106 for display on the screen 122. 

30 [034] When the sports simulation module option 406 is selected, the 

computer 104 presents a sports module menu listing available sports modules that can 
be selected (step 306). In this example, the sports module menu includes a baseball 
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module 420, a football module 422, a soccer module 424, a golf module 426, a 
hockey module 428, a skiboard module 430 and an other sport option 432. When a 
sports module is selected, the computer 1 04 presents an operation mode menu and 
prompts the player to select a mode of operation (step 308). The available modes of 
5 operation include a training mode, a practice mode and a play game mode. The 
computer 104 also downloads the characteristic projectile signature associated with 
the selected sports module to the on-board processors of the digital cameras 130. 
[035] When the training mode is selected, the computer 104 executes the 

training routine associated with the selected sports module. During execution of the 
1 0 training routine, a training menu 440 is presented to the player allowing the player to 
select a training lesson (step 3 1 0). The training lessons cover various elements of the 
particular selected sport and include video clips that provide tips from professional 
athletes. Once the training lesson has been selected, the training lesson is commenced 
(step 312). 

1 5 [036] When the practice mode is selected, the computer 104 executes the 

practice routine associated with the selected sports module. During execution of the 
practice routine, a practice menu 442 is presented to the player allowing the player to 
select a practice session (step 314). The practice sessions include skills oriented 
practice exercises associated with the particular selected sport. Once the practice 

20 session is selected a setup menu 444 is presented to the player that provides the player 
with the ability to set the skill level, preferences and the statistics to be maintained 
during practice (step 316). Once the practice session setup has been completed, the 
selected practice session is commenced (step 318). 

[037] When the play game mode is selected, the computer 104 executes the 

25 play game routine associated with the selected sports module. During execution of 
the play game routine, a game menu 450 is presented to the player allowing the player 
to select a game session (step 320). For example, if the golf module 426 is selected at 
step 306, the game menu 450 presents a list of different golf courses that can be 
played. Once the game session is selected a setup menu 452 is presented to the player 
30 that provides the player with the ability to set the skill level, preferences and the 

statistics to be maintained during game play (step 322). Once the game session setup 
has been completed, the selected game is commenced (step 324). 



-13- 



[038] During training, practice or game play, the computer 104 outputs video 

image data to the projector 106 causing the projector 106 to project a video sequence 
portraying a visually apparent three-dimensional sports scene on the display surface 
124 that includes a target at which the projectile is to be launched (step 500). The 
5 computer 104 also conditions the digital cameras 130 to capture a background image 
of the projectile tracking region 120 devoid of a projectile (step 502) and then scan 
the projectile tracking region to look for the presence of a launched projectile at a 
very high frame rate in the order of 360Hz (step 504). The player is then prompted to 
launch the projectile at the screen 122 (step 506). 

10 [039] When the player launches the projectile at the projectile tracking 

apparatus 102 and the projectile enters the projectile tracking region 120, the digital 
cameras 130 scanning the projectile tracking region 120 detect the presence of the 
projectile. In response, the digital cameras 130 synchronously capture a series of 
images of the projectile as it travels through the projectile tracking region at a 

15 decreased frame rate in the order of 120Hz (step 508). As each image is captured, the 
on-board processor of the digital camera 130 segments and analyzes the image data to 
detect the center of mass of the projectile and determine the two-dimensional 
rectangular coordinates of the center of mass relative to the field of view FOV of the 
digital camera (step 510). The rectangular coordinates are then conveyed to the 

20 computer 104 via the high-speed serial data link. As a result, the computer 104 

receives a series of two-dimensional rectangular coordinates from each digital camera 
130 representing the two-dimensional positions of the projectile as it travels through 
the projectile tracking region 120 relative to that digital camera 130. 
[040] At step 502, each digital camera 130 captures a background image and 

25 generates a reference greyscale histogram. Since the projectile tracking region 120 is 
illuminated by IR radiation, when no projectile is in the projectile tracking region 120, 
the pixels of the digital cameras 1 30 are all illuminated. However, when a projectile 
enters the projectile tracking region 120, the projectile occludes IR illumination and 
appears as a cluster of activated (i.e. dark) pixels in the captured images. The digital 

30 cameras 1 30 are then conditioned to capture images at a first frame rate and generate 
corresponding histograms. The generated histograms are compared with the reference 
greyscale histogram to yield a difference histogram. Each difference histogram is 
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examined to detect pixel clusters having a signature that corresponds to the stored 
characteristic projectile signature. If a pixel cluster in the difference histogram 
matches the characteristic projectile signature, the captured image is deemed to 
include an image of the projectile. The background image is updated before each 
5 projectile launch to take ambient lighting conditions into account and maintain 
accurate projectile detection 

[041] Upon receipt of the two-dimensional rectangular coordinates from the 

digital cameras 130, the computer 104 in turn calculates the positions of the 
projectile's center of mass in three-dimensional space using triangulation throughout 

1 0 its travel through the projectile tracking region 1 20 (step 520). With the position of 
the projectile in three-dimensional space known during its travel through the 
projectile tracking region 120 and knowing the frame rates of the digital cameras 130, 
the computer 104 calculates the velocity of the projectile over each image frame (step 
522). The computer 104 then compares each calculated velocity with the previously 

1 5 calculated velocity to determine the deceleration/acceleration of the projectile and 
calculates the average velocity of the projectile over the entire projectile tracking 
region 120 (step 524). 

[042] With the three-dimensional positions, average velocity and 

deceleration/acceleration of the projectile known over the projectile tracking region 

20 120, the computer 104 extrapolates an accurate trajectory for the projectile allowing a 
realistic simulation of curved and/or arcing projectiles to be generated (step 526). The 
computed projectile trajectory is then used to determine a sports result by estimating 
the intersection of the calculated projectile trajectory with the displayed video target 
(step 528). With the projectile trajectory computed and the sports result determined, 

25 the computer 104 updates the image data that is conveyed to the projector 106 so that 
the video sequence displayed on the display surface 124 of the screen 122 shows the 
simulated flight of the projectile and the sports result (step 530). 
[043] During video sequence display, when a simulation of the projectile 

flight is shown a graphical duplicate of the projectile is projected onto the display 

30 surface 124 of the screen 122 that begins its flight from the impact point of the 

projectile with the screen 122. In this manner, the projectile appears to continue its 
trajectory into the video scene thereby to achieve a realistic video effect. 
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[044] Depending on the selected sports module, the computer 104 will 

update the image data so that foreground action elements react to the launched 
projectile and portray an appropriate response. For example, if the hockey module is 
selected, the player is presented with a video scene showing a foreground action 
5 goaltender defending a background net. In this case, the display adapter assembly 
configures the buffers so that the background image data is stored in an 8-bit buffer, 
the puck projectile data is stored in an 8-bit buffer and foreground action goaltender 
data is stored in a 16-bit buffer. The computer 104 updates the video sequence in 
response to a launched puck travelling through the projectile tracking region 120 so 

1 0 that the goaltender moves in the appropriate direction to stop the launched puck. In 
this case, the set skill level and the trajectory and velocity of the launched puck 
determines whether the goaltender moves quickly enough to stop the launched puck. 
As will be appreciated, when other sports modules are selected, the display adapter 
assembly configures the buffers to best generate a realistic video sequence. 

15 [045] If desired, the sports simulation system 100 may include a projectile 

delivery device to deliver a projectile into the player area A for subsequent launching 
of the projectile by the player towards the projectile tracking apparatus 102. For 
example, in the case of baseball, the sports simulation system may include a pitching 
machine to pitch baseballs towards the player allowing the player to hit baseballs 

20 towards the projectile tracking apparatus 102. In the case of hockey, the sporty 
simulation system 100 may including a puck passing device to pass pucks to the 
player allowing the player to shoot the passed pucks towards projectile tracking 
apparatus 102. Depending on the nature of the projectile, if desired, the sports 
simulation system may be surrounded by protective netting or enclosed within a 

25 theatre to inhibit errant projectiles from escaping. 

[046] Also, if desired, the computer 104 may store a sports rules reference 

library and/or a statistics library associated with each of the sports modules. In this 
case, the libraries are accessible during game play through pop-up menus presented 
on the display surface 124 to provide the player with relevant sports related data. 

30 [047] Turning now to Figure 1 1, an alternative digital camera 130' and 

mirror 128' arrangement for the top corners of the projectile tracking apparatus is 
shown. As can be seen, in this arrangement, each digital camera 130' looks through a 
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transparent plastic window 600 in the side post 112' and through a transparent plastic 
window 602 in the crossbar 1 14'. The mirror 128' overlies the window 602 to aim 
the optical focal axis of the digital camera 130' toward the center of the display 
surface. 

5 [048] The projectile tracking apparatus 102 has been described as including 

four digital cameras 130, each looking into the projectile tracking region 120 from a 
different corner of the frame 110. Those of skill in the art will appreciate that only 
one pair of digital cameras having overlapping fields of view is required to triangulate 
the position of the projectile as it travels through the projectile tracking region 120. 

10 Using only one pair of digital cameras is generally suitable in environments where the 
projectile is launched at slower speeds. When only one pair of digital cameras 1 30 is 
used, if the projectile typically enters the projectile tracking region 120 adjacent the 
lower portion of the frame 1 10, it is preferred that the fields of view of the digital 
cameras 130 look down into the projectile tracking region 120 from the top corners of 

15 the frame 110. If the projectile typically enters the projectile tracking region 120 
adjacent the upper portion of the frame 1 10, it is preferred that the fields of view of 
the digital cameras 130 look up into the projectile tracking region 120 from the 
bottom corners of the frame 110. Depending on the sport being simulated and the 
speed of the projectile, the digital cameras 130 may be conditioned to capture multiple 

20 exposures during a single image frame in order to ensure sufficient light is gathered 
when the digital cameras are operated at very high frame rates: In addition, the IR 
LED arrays may be strobed. 

[049] Although the sports simulation system 100 has been described as 

including a ceiling mounted front projector 106 in combination with a screen 122, 

25 those of skill in the art will appreciate that alternative projection devices may be used. 
For example, a rear video projector may be used to project images onto the rear 
surface of the display screen 122. The projector and display screen may alternatively 
be replaced with a display unit such as a cathode ray tube, a plasma display or a liquid 
crystal display, that^receives video image output from the computer 104. In this case, 

30 the display unit is preferably disposed behind a transparent protective panel. 

[050] In the preferred embodiment, the projectile tracking apparatus has been 

described as determining the existence of the projectile in a captured image by 
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detecting clusters of dark pixels. Those of skill in the art will appreciate however that 
the projectile may also be determined by detecting clusters of white pixels resulting 
from illumination reflecting off of the projectile. Those of skill in the art will also 
appreciate that the number of processing stages may be increased or decreased as 
5 desired to handle processing of the digital camera image data effectively in real-time 
and provide a realistic projectile simulation. 

[051] Although preferred embodiments of the present invention have been 

described, those of skill in the art will appreciate that variations and modifications 
may be made without departing from the spirit and scope thereof as defined by the 
10 appended claims. 



